Skip to content

feat(api): add POST /v1/images/upscale endpoint#10227

Open
petechentw wants to merge 2 commits into
mudler:masterfrom
shihyunhuang:feature/image-upscale-api
Open

feat(api): add POST /v1/images/upscale endpoint#10227
petechentw wants to merge 2 commits into
mudler:masterfrom
shihyunhuang:feature/image-upscale-api

Conversation

@petechentw

Copy link
Copy Markdown
Contributor

Summary

Adds a new `/v1/images/upscale` endpoint for upscaling images via LocalAI backends.

  • Accepts `multipart/form-data` with `model`, `image`, and `scale` fields
  • Returns an `OpenAIResponse`-compatible JSON with the upscaled image URL
  • Supports dedicated upscaler models (e.g. RealESRGAN via diffusers)
  • Falls back to high-quality Lanczos resize when no upscaler pipeline is loaded
  • Gated under `FeatureImages` (same as `/v1/images/generations`)

Changes

Layer File Change
Proto `backend/backend.proto` Add `UpscaleImage` RPC + `UpscaleImageRequest` message
gRPC `pkg/grpc/{interface,client,server,embed}.go` Implement `UpscaleImage` across all gRPC layers
gRPC Base `pkg/grpc/base/base.go` Add `UpscaleImage` stub so all backends satisfy the interface
Backend `core/backend/upscale.go` New backend helper (mirrors `ImageGeneration`)
Endpoint `core/http/endpoints/openai/upscale.go` Multipart form handler with Swagger doc
Route `core/http/routes/openai.go` Register `POST /v1/images/upscale` and `/images/upscale`
Auth `core/http/auth/features.go` Gate under `FeatureImages`
Python `backend/python/diffusers/backend.py` `UpscaleImage` method with diffusers + Lanczos fallback

Usage

```bash
curl -X POST http://localhost:8080/v1/images/upscale
-F model=realesrgan
-F scale=4
-F image=@input.png
```

Test

Verified locally: server starts and `POST /v1/images/upscale` responds with `400` (missing params) instead of `404`, confirming the route is correctly registered.

Add a new image upscaling endpoint that accepts a source image and
returns an upscaled version. Supports selectable upscaler models
(e.g. realesrgan) and a configurable scale factor (2x or 4x).

- backend.proto: add UpscaleImage RPC and UpscaleImageRequest message
- pkg/grpc: implement UpscaleImage in Backend interface, client, server
  and embed shim
- core/backend/upscale.go: new backend helper (mirrors ImageGeneration)
- core/http/endpoints/openai/upscale.go: new multipart/form-data handler
- core/http/routes/openai.go: register POST /v1/images/upscale
- core/http/auth/features.go: gate upscale routes under FeatureImages
- backend/python/diffusers/backend.py: implement UpscaleImage — uses
  diffusers upscale pipeline when loaded, falls back to Lanczos resize
All Go backends embedding Base now satisfy the AIModel interface
without needing to implement UpscaleImage explicitly.
@mudler

mudler commented Jun 9, 2026

Copy link
Copy Markdown
Owner

@petechentw that's a good addition. thanks. Two things:

  • Could you add a model gallery that makes use of this so this can be functional already?
  • Please run git commit --amend --signoff so DCO check passes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants